Restore pluggable email backend for task failure/retry alerts#69877
Merged
Conversation
eladkal
reviewed
Jul 14, 2026
potiuk
force-pushed
the
restore-pluggable-email-backend
branch
2 times, most recently
from
July 20, 2026 10:24
2f222f9 to
2e92be7
Compare
potiuk
marked this pull request as ready for review
July 20, 2026 10:31
potiuk
force-pushed
the
restore-pluggable-email-backend
branch
2 times, most recently
from
July 20, 2026 10:49
d3c1bcc to
fd671dc
Compare
amoghrajesh
reviewed
Jul 20, 2026
potiuk
force-pushed
the
restore-pluggable-email-backend
branch
from
July 20, 2026 10:59
fd671dc to
816add6
Compare
amoghrajesh
approved these changes
Jul 20, 2026
amoghrajesh
left a comment
amoghrajesh
left a comment
Contributor
There was a problem hiding this comment.
Looks good after the comments are handled.
kosteev
approved these changes
Jul 20, 2026
potiuk
force-pushed
the
restore-pluggable-email-backend
branch
from
July 20, 2026 11:35
816add6 to
0ff8364
Compare
Since apache#57354, task email_on_failure / email_on_retry alerts were routed unconditionally through SmtpNotifier, silently ignoring the [email] email_backend configuration. Custom backends (SES, SendGrid, org-internal) stopped delivering failure/retry emails even though the deprecated email_on_* parameters still worked. This restores the old behaviour using the existing [email] email_backend option -- no new configuration is introduced: - A non-default [email] email_backend is transparently wrapped in a new LegacyEmailBackendNotifier, so existing SES / SendGrid / custom backends keep delivering alerts unchanged. The backend is resolved from config at notify time, so the Task SDK keeps no static dependency on airflow.utils.email. - Otherwise the default SmtpNotifier is used, exactly as before. The notifier lives in the Task SDK (airflow.sdk.execution_time.email_backend) next to its only caller, so no extra provider needs to be installed for a custom email backend to keep working. Both failure-email entry points (the worker task-runner path and the DAG-processor callback path) funnel through the same function, so the selected backend is used consistently regardless of how the task failed. The deprecated email_on_* parameters are not un-deprecated; this only keeps their existing behaviour pluggable until removal in Airflow 4.
potiuk
force-pushed
the
restore-pluggable-email-backend
branch
from
July 20, 2026 11:44
0ff8364 to
4d0e66d
Compare
Contributor
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
github-actions Bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Jul 20, 2026
…y alerts (apache#69877) Since apache#57354, task email_on_failure / email_on_retry alerts were routed unconditionally through SmtpNotifier, silently ignoring the [email] email_backend configuration. Custom backends (SES, SendGrid, org-internal) stopped delivering failure/retry emails even though the deprecated email_on_* parameters still worked. This restores the old behaviour using the existing [email] email_backend option -- no new configuration is introduced: - A non-default [email] email_backend is transparently wrapped in a new LegacyEmailBackendNotifier, so existing SES / SendGrid / custom backends keep delivering alerts unchanged. The backend is resolved from config at notify time, so the Task SDK keeps no static dependency on airflow.utils.email. - Otherwise the default SmtpNotifier is used, exactly as before. The notifier lives in the Task SDK (airflow.sdk.execution_time.email_backend) next to its only caller, so no extra provider needs to be installed for a custom email backend to keep working. Both failure-email entry points (the worker task-runner path and the DAG-processor callback path) funnel through the same function, so the selected backend is used consistently regardless of how the task failed. The deprecated email_on_* parameters are not un-deprecated; this only keeps their existing behaviour pluggable until removal in Airflow 4. (cherry picked from commit f7dec02) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
aws-airflow-bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Jul 20, 2026
…y alerts (apache#69877) Since apache#57354, task email_on_failure / email_on_retry alerts were routed unconditionally through SmtpNotifier, silently ignoring the [email] email_backend configuration. Custom backends (SES, SendGrid, org-internal) stopped delivering failure/retry emails even though the deprecated email_on_* parameters still worked. This restores the old behaviour using the existing [email] email_backend option -- no new configuration is introduced: - A non-default [email] email_backend is transparently wrapped in a new LegacyEmailBackendNotifier, so existing SES / SendGrid / custom backends keep delivering alerts unchanged. The backend is resolved from config at notify time, so the Task SDK keeps no static dependency on airflow.utils.email. - Otherwise the default SmtpNotifier is used, exactly as before. The notifier lives in the Task SDK (airflow.sdk.execution_time.email_backend) next to its only caller, so no extra provider needs to be installed for a custom email backend to keep working. Both failure-email entry points (the worker task-runner path and the DAG-processor callback path) funnel through the same function, so the selected backend is used consistently regardless of how the task failed. The deprecated email_on_* parameters are not un-deprecated; this only keeps their existing behaviour pluggable until removal in Airflow 4. (cherry picked from commit f7dec02) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
dabla
pushed a commit
to dabla/airflow
that referenced
this pull request
Jul 21, 2026
…ache#69877) Since apache#57354, task email_on_failure / email_on_retry alerts were routed unconditionally through SmtpNotifier, silently ignoring the [email] email_backend configuration. Custom backends (SES, SendGrid, org-internal) stopped delivering failure/retry emails even though the deprecated email_on_* parameters still worked. This restores the old behaviour using the existing [email] email_backend option -- no new configuration is introduced: - A non-default [email] email_backend is transparently wrapped in a new LegacyEmailBackendNotifier, so existing SES / SendGrid / custom backends keep delivering alerts unchanged. The backend is resolved from config at notify time, so the Task SDK keeps no static dependency on airflow.utils.email. - Otherwise the default SmtpNotifier is used, exactly as before. The notifier lives in the Task SDK (airflow.sdk.execution_time.email_backend) next to its only caller, so no extra provider needs to be installed for a custom email backend to keep working. Both failure-email entry points (the worker task-runner path and the DAG-processor callback path) funnel through the same function, so the selected backend is used consistently regardless of how the task failed. The deprecated email_on_* parameters are not un-deprecated; this only keeps their existing behaviour pluggable until removal in Airflow 4.
potiuk
added a commit
that referenced
this pull request
Jul 21, 2026
…y alerts (#69877) (#70129) Since #57354, task email_on_failure / email_on_retry alerts were routed unconditionally through SmtpNotifier, silently ignoring the [email] email_backend configuration. Custom backends (SES, SendGrid, org-internal) stopped delivering failure/retry emails even though the deprecated email_on_* parameters still worked. This restores the old behaviour using the existing [email] email_backend option -- no new configuration is introduced: - A non-default [email] email_backend is transparently wrapped in a new LegacyEmailBackendNotifier, so existing SES / SendGrid / custom backends keep delivering alerts unchanged. The backend is resolved from config at notify time, so the Task SDK keeps no static dependency on airflow.utils.email. - Otherwise the default SmtpNotifier is used, exactly as before. The notifier lives in the Task SDK (airflow.sdk.execution_time.email_backend) next to its only caller, so no extra provider needs to be installed for a custom email backend to keep working. Both failure-email entry points (the worker task-runner path and the DAG-processor callback path) funnel through the same function, so the selected backend is used consistently regardless of how the task failed. The deprecated email_on_* parameters are not un-deprecated; this only keeps their existing behaviour pluggable until removal in Airflow 4. (cherry picked from commit f7dec02) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Restore the ability to deliver
email_on_failure/email_on_retrytask alerts through theconfigured
[email] email_backend(SES, SendGrid, or any custom callable), which regressedin #57354.
Why
email_on_failure/email_on_retryare deprecated (slated for Airflow 4, migrate toSmtpNotifier) but still functional. Since #57354, task failure/retry emails are routedunconditionally through
SmtpNotifier, silently ignoring the[email] email_backendconfiguration. Deployments using a custom
email_backend(Amazon SES, SendGrid,org-internal) stopped receiving failure/retry alerts even though the config option still
exists and the deprecated params still work. This restores that behaviour while keeping the
single-notifier design #57354 introduced.
How
_send_error_email_notification(shared by both the worker task-runner path and theDAG-processor callback path) resolves the delivery mechanism from the existing
[email] email_backendoption — no new configuration is added:[email] email_backendis transparently wrapped in a newLegacyEmailBackendNotifier(airflow.sdk.execution_time.email_backend), so existingSES/SendGrid setups keep working unchanged. The backend is resolved from config at notify
time, so the Task SDK keeps no static dependency on
airflow.utils.email.SmtpNotifieris used, exactly as today.The notifier lives in the Task SDK next to its only caller, so no additional provider needs
to be installed for a custom email backend to keep working.
Because both failure-email entry points funnel through the same function, the selected
backend is used consistently regardless of how the task failed (normal failure vs.
zombie/killed).
Changes after review
An earlier revision of this PR added a new
[email] email_notifierconfig option. Pereladkal's review comment,that option has been dropped — the goal is to complete the Core→SMTP migration and
deprecate email settings in core, not add new ones. The fix now rides entirely on the
email_backendoption that already exists, andconfig.ymlis untouched by this PR.Tests
TestEmailNotifications(task-sdk): default still usesSmtpNotifier(unchangedregression tests), custom
email_backendwrapped and invoked with the rendered fields,unresolvable
email_backendlogged (no raise).TestExecuteEmailCallbacks(DAG-processor path): unchanged and green — confirms bothentry points stay consistent.
test_email_backend.py(task-sdk): unit tests forLegacyEmailBackendNotifierdispatch.Notes
email_on_*params are not un-deprecated; this only keeps theirexisting behaviour pluggable until removal in Airflow 4.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines